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(57) [Abstract] 

[Problem] To automatically recognize the character 
information in an image, save only the character 
information as an index, provide it to an image storage 
and playback terminal, and increase administrative 
simplicity and convenience. 

[Solution means] In character information storage 
terminals 2 and 3, by inputting an image, extracting its 
character information, and storing as index information 
for image viewing only the character information resulting 
from the extracting, the need for storing image 
information is eliminated, the storage volume is reduced, 
image storage processing is eliminated, and the burden on 
the equipment is lessened. And by sending only the 
character information from the character information 
storage terminals 2 and 3 to the image storage and 
playback terminals 6 and 7 and not sending the image 
information, a system is realized simply, with no need for 
a highband [sic; homophone misprint for 'broadband'] 
communications circuit 9. And by the character 
information storage terminals 2 and 3 and the image 
storage and playback terminals 6 and 7 synchronizing 
their time information by periodically referencing the time 
information server terminal 10, the time discrepancy 
between the time of the requested character information 
and the time of the provided character information is 
eliminated. 
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[Claims] 

[Claim 1] A character information provision system 
that is characterized in that it has a character information 
storage terminal that extracts character information 
concerning the image from an image input from an image 
input source and stores in association said extracted 
character information and time information on when said 
extraction took place, and 

an image storage and playback terminal that stores an 
image input from said image input source and time 
information on when said image was input, and plays 
back said stored image information; 

said character information storage terminal and said 
image storage and playback terminal have a time 
synchronization means that synchronizes the time 
information; 

said character information storage terminal retrieves 
said stored character information from the time 
information of the image for playback on said image 
storage and playback terminal and transmits it to said 
image storage and playback terminal; and 

said image storage and playback terminal sends the 
time information of the image for playback to said 
character information storage terminal and references the 
character information received from said character 
information storage terminal as an index when playing 
back said image. 

[Claim 2] In a character information storage terminal, 

a character information provision method that is 
characterized in that it has a time synchronization 
procedure that synchronizes the time information with the 
image storage and playback terminal that stores images 
for playback, 

a character information extraction procedure that 
extracts character information concerning an image from 
said images for playback, 

a character information storage procedure that stores in 
association said extracted character information and said 
time information on when said extraction took place, 

a time information communication procedure that 
receives from the image storage and playback terminal the 
time information of said image for playback, 

a character information retrieval procedure that 
retrieves from among said stored character information 
the character information that has said received time 
information, and 

a character information communication means that 
transmits said retrieved character information to said 
image storage and playback terminal. 

[Claim 3] Being a character information provision 
method in the case in which there are two or more image 
storage and playback terminals, or character information 
storage terminals, or both, a character information 
provision method as described in claim 2 that is 
characterized in that in the character information storage 
procedure, in addition to character information and time 
information concerning the playback image, attribute 
information concerning the terminal is also stored; 

in the time information communication procedure, in 
addition to the time information, attribute information 



concerning the terminal is also received from the image 
storage and playback terminal; and 

in the character information retrieval procedure, 
character information is retrieved that corresponds to said 
received time information and attribute information 
concerning the terminal. 

[Claim 4] In an image storage and playback terminal, 

a character information provision method that is 
characterized in that it has a time synchronization 
procedure that synchronizes the time information with the 
character information storage terminal that stores 
character information concerning images for playback, 

an image storage procedure that stores in association 
said image for playback and said time information, 

a time information communication procedure that 
transmits to said character information storage terminal 
the time information associated with said image for 
playback, 

a character information communication procedure that 
receives from said character information storage terminal 
the character information extracted from said image for 
playback retrieved based on said transmitted time 
information, and 

an index reference procedure that references said 
received character information as an index when playing 
back said image for playback. 

[Claim 5] Being a character information provision 
method in the case in which there are two or more 
character information storage terminals, or image storage 
and playback terminals, or both, 

a character information provision method as described 
in claim 4 that is characterized in that in the image storage 
procedure, in addition to the image for playback and time 
information to be stored, attribute information concerning 
the terminal is also stored; 

in the time information communication procedure, in 
addition to the time information to be transmitted to the 
character information storage terminal, attribute 
information concerning said terminal is also transmitted; 
and 

in the character information communication procedure, 
in addition to said time information, character information 
that corresponds to the attribute information concerning 
said terminal is also received. 

[Claim 6] In said time synchronization procedure, 

a character information provision method as described 
in any of claims 2 to 4 that is characterized in that 
between the character information storage terminal and 
the image storage and playback terminal, one terminal 
periodically references the time information of the other 
terminal and sets the referenced time information as its 
own terminal's time information. 

[Claim 7] In a character information provision method 
in the case in which there are two or more character 
information storage terminals, or image storage and 
playback terminals, or both, 

a character information provision method as described 
in any of claims 2 to 4 that is characterized in that in the 
time synchronization procedure, 

time information is required for a terminal that will 
serve as the standard for time information among two or 
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more character information storage terminals or image 
storage and playback terminals, the terminals other than 
said terminal that serves as the standard receive time 
information from said terminal that serves as the standard, 
and said received time information is set as the time 
information for the terminals themselves. 

[Claim 8] In a character information provision method 
in the case in which there are two or more character 
information storage terminals, or image storage and 
playback terminals, or both, 

a character information provision method as described 
in any of claims 2 to 4 that is characterized in that in the 
time synchronization procedure, 

the terminal that serves as the standard for time 
information transmits time information for 
synchronization to the terminals other than said terminal 
that serves as the standard. 

[Claim 9] A recording on which is recorded a program 
that realizes a character information provision method that 
is characterized in that the procedure in the character 
information provision method as described in any of 
claims 2 to 8 

is recorded, as a program to be executed on a 
computer, on a recording medium that can be read by said 
computer. 

[Detailed Description of the Invention] 

[0001] 

[Field of technology to which the invention belongs] 
This invention concerns technology for extracting 
character information from image information such as for 
example what is distributed by television broadcasting 
stations, and storing and providing the extracted results as 
index information for playing back images. 

[0002] 

[Prior art] Technology has previously been proposed 
for extracting character information such as on-screen 
titles ['telops'] from images, using the extracted results as 
an index, and playing back the images. 

[0003] Patent application H9-334458 [1997] describes 
technology in which, in terminals like computers and 
television that have an image input function, frame images 
on which are displayed characters and which are displayed 
in the midst of an image that has been input are 
automatically detected, and the detected frame image is 
displayed on the screen as memo information. 

[0004] And patent application H9-335770 [1997] 
describes technology in which images are keyword- 
searched using character code information obtained by 
automatically recognizing character information in the 
frame image detected from a video image, while patent 
application H9-340134 [1997] further describes 
technology in which the character code that is obtained is 
used to retrieve information on the Internet and acquire 
Internet information relating to an input image. 

[0005] And with regard to technology for 
automatically recognizing the audio information in a 
video image, converting the audio into character 
information, and searching for an image by keyword input 
or speech input, such technology has been realized in a 
computer software product called a cross-media editor by 



the Media Drive company, with a computer terminal that 
runs the software. 

[0006] Also, patent application H9-334459 [1997] 
describes technology for an image provision method in 
which recognition of the characters in a video image is 
carried out automatically, the recognized results are used 
as an index for image playback, and image information is 
distributed over a communication circuit upon request 
from an image storage and playback terminal. 

[0007] 

[Problems that the invention is to solve] But in all of 
the prior art referred to above, there has been the problem 
that it is necessary to have in the terminal a storage unit 
having enormous storage capacity, because the character 
information or audio information is automatically 
recognized on the terminal into which the image is input, 
the recognition results are stored, and the image 
information that is input must also be stored; in addition, 
there is much character recognition, voice recognition, 
and image storage processing, which imposes a big 
burden on the terminal. 

[0008] And in the image information provision 
technology via communication circuit that is described in 
patent application H9-334459 [1997], there has been the 
problem that it is necessary to distribute through the 
communication circuit not just the character information 
but the image information as well, which requires a broad 
band for the communication circuit. 

[0009] Thus in this invention, in light of the problems 
of the above prior art, the task is to provide a character 
information provision method in which the character 
information in an image is automatically recognized, only 
the recognized results are saved as index data for image 
viewing, and they are provided to the user's terminal, in a 
way that makes the operation and management in the 
character information provision method simpler and more 
convenient. 

[0010] A further task taken up by this invention is to 
provide a character information provision system with a 
server-client configuration, with the provision side 
providing index information based on the character 
recognition results, and the using side using the index 
information to view images. 

[0011] 

[Means of solving the problems] In order to solve the 
above problems, the character information provision 
system according to this invention is characterized in that 
it has a character information storage terminal that 
extracts character information concerning the image from 
an image input from an image input source and stores in 
association said extracted character information and time 
information on when said extraction took place, and an 
image storage and playback terminal that stores an image 
input from said image input source and time information 
on when said image was input, and plays back said stored 
image information; said character information storage 
terminal and said image storage and playback terminal 
have a time synchronization means that synchronizes the 
time information; said character information storage 
terminal retrieves said stored character information from 
the time information of the image for playback on said 
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image storage and playback terminal and transmits it to 
said image storage and playback terminal; and said image 
storage and playback terminal sends the time information 
of the image for playback to said character information 
storage terminal and references the character information 
received from said character information storage terminal 
as an index when playing back said image. 

[0012] Likewise, the character information provision 
method according to this invention is characterized in that 
in a character information storage terminal, it has a time 
synchronization procedure that synchronizes the time 
information with the image storage and playback terminal 
that stores images for playback, a character information 
extraction procedure that extracts character information 
concerning an image from said images for playback, a 
character information storage procedure that stores in 
association said extracted character information and said 
time information on when said extraction took place, a 
time information communication procedure that receives 
from the image storage and playback terminal the time 
information of said image for playback, a character 
information retrieval procedure that retrieves from among 
said stored character information the character 
information that has said received time information, and a 
character information communication means that 
transmits said retrieved character information to said 
image storage and playback terminal. 

[0013] Or, being said character information provision 
method in the case in which there are two or more image 
storage and playback terminals, or character information 
storage terminals, or both, it is characterized in that in the 
character information storage procedure, in addition to 
character information and time information concerning 
the playback image, attribute information concerning the 
terminal is also stored; in the time information 
communication procedure, in addition to the time 
information, attribute information concerning the terminal 
is also received from the image storage and playback 
terminal; and in the character information retrieval 
procedure, character information is retrieved that 
corresponds to said received time information and 
attribute information concerning the terminal. 

[0014] Or, in an image storage and playback terminal, 
it is characterized in that it has a time synchronization 
procedure that synchronizes the time information with the 
character information storage terminal that stores 
character information concerning images for playback, an 
image storage procedure that stores in association said 
image for playback and said time information, a time 
information communication procedure that transmits to 
said character information storage terminal the time 
information associated with said image for playback, a 
character information communication procedure that 
receives from said character information storage terminal 
the character information extracted from said image for 
playback retrieved based on said transmitted time 
information, and an index reference procedure that 
references said received character information as an index 
when playing back said image for playback. 

[0015] Or, being a character information provision 
method in the case in which there are two or more 



character information storage terminals, or image storage 
and playback terminals, or both, it is characterized in that 
in the image storage procedure, in addition to the image 
for playback and time information to be stored, attribute 
information concerning the terminal is also stored; in the 
time information communication procedure, in addition to 
the time information to be transmitted to the character 
information storage terminal, attribute information 
concerning said terminal is also transmitted; and in the 
character information communication procedure, in 
addition to said time information, character information 
that corresponds to the attribute information concerning 
said terminal is also received. 

[0016] Or, in said character information provision 
method, it is characterized in that, in said time 
synchronization procedure, between the character 
information storage terminal and the image storage and 
playback terminal, one terminal periodically references 
the time information of the other terminal and sets the 
referenced time information as its own terminal's time 
information. 

[0017] Or, in said character information provision 
method in the case in which there are two or more 
character information storage terminals, or image storage 
and playback terminals, or both, it is characterized in that 
in the time synchronization procedure, 

time information is required for a terminal that will 
serve as the standard for time information among two or 
more character information storage terminals or image 
storage and playback terminals, the terminals other than 
said terminal that serves as the standard receive time 
information from said terminal that serves as the standard, 
and said received time information is set as the time 
information for the terminals themselves. 

[0018] Or, in a character information provision 
method in the case in which there are two or more 
character information storage terminals, or image storage 
and playback terminals, or both, it is characterized in that 
in the time synchronization procedure, the terminal that 
serves as the standard for time information transmits time 
information for synchronization to the terminals other 
than said terminal that serves as the standard. 

[0019] Or, it is characterized in that the procedures in 
said character information provision method are recorded, 
as programs to be executed on a computer, on a recording 
medium that can be read by said computer. 

[0020] With this invention, in a character information 
storage terminal, by inputting an image, extracting its 
character information, and storing in a device as index 
information for image viewing only the character code 
information resulting from the extracting, the need for 
storing image information input as in the aforementioned 
prior art is eliminated, 

a reduction in the capacity of the storage part in the 
device as compared with the prior art is realized, and in 
addition one may forgo image storage processing, thereby 
lessening the burden on the devices from what it would be 
under the aforesaid prior art. 

[0021] Also, with this invention, the only information 
that need be sent from the character information storage 
terminal to the image storage and playback terminal is 
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character information, with no need to send image 
information, thus making it possible to simply realize a 
system that does not require the highband [sic; misprint 
for 'broadband'] communication circuit that is needed 
with the aforesaid prior art. 

[0022] Also, unlike patent application H9-334459, 
with this invention the terminal that extracts the character 
information in an image exists separately from the 
terminal that plays back the image, but by synchronizing 
the time information between the two terminals by a time 
synchronization procedure, it is made possible to provide 
character information that matches a request from an 
image storage and playback terminal with no time 
discrepancy arising between the time of the character 
information requested from the image storage and 
playback terminal and the time of the character 
information provided from the character information 
storage terminal to the image storage and playback 
terminal. 

[0023] 

[Embodiments of the invention] In the following, we 
describe embodiments of this invention with reference to 
the drawings. 

[0024] Figure 1 is a block diagram showing the 
schematic composition of the character information 
provision system of the first embodiment of this 
invention. 

[0025] The character information provision system of 
this embodiment consists of character information storage 
terminals 2 and 3 and image storage and playback 
terminals 6 and 7, to which are connected TV tuners 1, 4, 
5, and 8, which receive television broadcasts and output 
image information, and a communications circuit 9, which 
connects the character information storage terminals 2 and 
3, the image storage and playback terminals 6 and 7, and a 
time information server terminal 10. 

[0026] The character information storage terminals 2 
and 3 and the image storage and playback terminals 6 and 
7 are connected periodically via a time information server 
terminal 10 and the communication circuit 9, and they 
reference the time information of the time information 
server terminal 10 and take it as their own time 
information. 

[0027] The character information storage terminals 2 
and 3 and the image storage and playback terminals 6 and 
7 input image information from the TV tuners 1, 4, 5, and 
8, extract character information from the image 
information input by the character information storage 
terminals 2 and 3 and store it, and the image information 
that is input is stored and played back by the image 
storage and playback terminals 6 and 7. 

[0028] The character information storage terminals 2 
and 3 and the image storage and the playback terminals 6 
and 7 are connected by the communication circuit 9, and 
when necessary for playing back images on the image 
storage and playback terminals 6 and 7, the time 
information for the images for playback is transmitted to 
the character information storage terminals 2 and 3 via the 
communication circuit 9, the character information 
storage terminals 2 and 3 transmit to the image storage 
and playback terminals 6 and 7, via the communication 



circuit 9, the character information associated with the 
time information that has been received, and the character 
information received by the image storage and playback 
terminals 6 and 7 is used as index information when 
playing back the images for playback, and the images for 
playback are played back. 

[0029] The TV tuners 1, 4, 5, and 8 as image input 
sources that are connected to the character information 
storage terminals 2 and 3 and the image storage and 
playback terminals 6 and 7 need not be TV tuners. For 
example, they may be computer terminals that receive 
broadcast images over the Internet, and in that case, a 
composition is adopted in which computer terminals 
instead of TV tuners are connected to the character 
information storage terminals 2 and 3 and the image 
storage and playback terminals 6 and 7. 

[0030] Also, if multiple channels of image 
information are input, such as in television broadcasting, a 
composition may be adopted in which the character 
information storage terminals 2 and 3 and the image 
storage and playback terminals 6 and 7 operate in parallel 
for the number of television broadcasting channels. 

[0031] Figure 2 is a timing chart for the time 
synchronization procedure showing the second 
embodiment of this invention. In the following charts, 
SOOO (where O is a digit) represents a processing step. 

[0032] As described in the first embodiment above, if 
the character information storage terminal, image storage 
and playback terminal, and time information server 
terminal are connected via a communication circuit, time 
synchronization alignment processing is periodically 
requested from the character information storage terminal 
and image storage and playback terminal (step 101), and a 
connection between the character information storage 
terminal and image storage and playback terminal and the 
time information server terminal is established via the 
communication circuit (step 102). 

[0033] After the connection is established, the time 
information server terminal transmits its own time 
information to the character information storage terminal 
and image storage and playback terminal in response to 
the time information request of each (step 103). 

[0034] Time information is transmitted from the time 
information server terminal, and the communication 
connection between the time information server terminal 
and the character information storage terminal and image 
storage and playback terminal is severed (step 1 04). 

[0035] The character information storage terminal and 
image storage and playback terminal set as their own time 
information the time information received from the time 
information server terminal (step 105). 

[0036] Figure 3 is a timing chart for the time 
synchronization procedure shown in the third embodiment 
of this invention. 

[0037] In a character information provision system 
consisting of one character information storage terminal 
and one image storage and playback terminal, if an image 
storage and playback terminal periodically performs time 
synchronization alignment taking the time information of 
the character information storage terminal as the standard, 
time synchronization alignment is requested from the 
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image storage and playback terminal (step 201), and a 
connection is established via the communication circuit 
between the image storage and playback terminal and the 
character information storage terminal (step 202). 

[0038] After the connection is established, the 
character information storage terminal transmits its own 
time information to the image storage and playback 
terminal in response to a time information request (step 
203). 

[0039] Time information is transmitted from the 
character information storage terminal, and the 
communication connection between the character 
information storage terminal and the image storage and 
playback terminal is severed (step 204). 

[0040] The image storage and playback terminal sets 
as its own time information the time information received 
from the character information storage terminal (step 
205). 

[0041] Figure 4 is a timing chart for the time 
synchronization procedure showing the fourth 
embodiment of this invention. 

[0042] In a character information provision system 
that consists of multiple terminals, two or more character 
information storage terminals, or image storage and 
playback terminals, or both, if the time information of one 
predetermined character information storage terminal or 
image storage and playback terminal (hereafter called 
simply a terminal) it taken as the standard and the other 
character information storage terminals and image storage 
and playback terminals periodically do a time 
synchronization alignment, then time synchronization 
alignment processing is requested from character 
information storage terminals and image storage and 
playback terminals other than the terminal that serves as 
the standard for time information (step 301), and a 
connection is established via the communication circuit 
between the character information storage terminals and 
image storage and playback terminals other than the 
terminal that serves as the standard for time information, 
and the terminal that serves as the standard for time 
information (step 302). 

[0043] After the connection is established, in response 
to time information requests the terminal that serves as the 
standard for time information transmits its own time 
information to the character information storage terminals 
and image storage and playback terminals other than the 
terminal that serves as the standard for time information 
(step 303). 

[0044] Time information is transmitted from the 
terminal that serves as the standard for time information, 
and the communication connection is severed between the 
terminal that serves as the standard for time information 
and the character information storage terminals and image 
storage and playback terminals other than the terminal that 
serves as the standard for time information (step 304). 

[0045] The character information storage terminals 
and image storage and playback terminals other than the 
terminal that serves as the standard for time information 
set as their own time information the time information 
received from the terminal that serves as the standard for 
time information (step 305). 



[0046] In order to synchronize with the time 
information of a given terminal, it suffices that it take its 
own time information as the standard and transmit its own 
time information to the other terminals. 

[0047] In the character information provision system 
of the first embodiment of this invention shown in Figure 
1, as the fifth embodiment of this invention we describe, 
with reference to the timing chart in Figure 5, the 
procedure up until character information is provided from 
the character information storage terminals 2 and 3 to the 
image storage and playback terminals 6 and 7 via the 
communication circuit 9. 

[0048] If, in the image storage and playback terminals 
6 and 7, the character information on the character 
information storage terminals 2 and 3 is to be received 
and images are to be viewed, then if the user performs an 
operation such as clicking the relevant button for an index 
request on the user interface in the image storage and 
playback terminals 6 and 7 (step 401), then the image 
storage and playback terminals 6 and 7 will be connected 
via the communication circuit 9 to the character 
information storage terminals 2 and 3 (step 402). 

[0049] The connection is established, and information 
on the starting time of the image information for playback 
stored in the image storage and playback terminals 6 and 7 
is transmitted to the character information storage 
terminals 2 and 3 (step 403). 

[0050] The character information storage terminals 2 
and 3 search among the stored character information for 
the character information associated with the starting time 
when the starting time was transmitted from the image 
storage and playback terminals 6 and 7 (step 404). At this 
time, the search results can be narrowed down to a 
specific one even if for example characters are displayed 
on multiple channels at the same time, by additionally 
saving an identifier by which the input source can be 
specified, such as for example the channel number, 
between the image storage and playback terminals 6 and 7 
and the character information storage terminals 2 and 3. 

[005 1] Upon receiving a character information request 
from the image storage and playback terminals 6 and 7, 
the character information storage terminals 2 and 3 
transmit the selected character information to the image 
storage and playback terminals 6 and 7 (step 405). 

[0052] When the image storage and playback 
terminals 6 and 7 receive this character information, the 
communication connection between the image storage and 
playback terminals 6 and 7 and the character information 
storage terminals 2 and 3 is severed (step 406). 

[0053] What is used for the time information to be 
transmitted from the image storage and playback terminals 
6 and 7 to the character information storage terminals 2 
and 3 is the time associated with the image information 
that one wants to play back in the image storage and 
playback terminals 6 and 7. For example, if dealing with 
television broadcasts, the composition may be one in 
which the broadcast starting time and ending time for 
each program are stored in the image storage and playback 
terminals 6 and 7 and the character information storage 
terminals 2 and 3, and if for the time information to be 
transmitted from the image storage and playback terminals 
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6 and 7 to the character information storage terminals 2 
and 3, any time is transmitted that lies between the 
broadcast starting time and broadcast ending time of 
program images that one wants to play back, then on the 
side of the character information storage terminals 2 and 
3, character information concerning a program that has a 
broadcast starting time and broadcast ending time in 
which the received time is included is transmitted to the 
image storage and playback terminals 6 and 7. 

[0054] This eliminates the need to set a strict 
synchronization between the image storage and playback 
terminals 6 and 7 and the character information storage 
terminals 2 and 3. 

[0055] Figure 6 is a processing flowchart for the 
character information storage terminals in a character 
information provision system according to the sixth 
embodiment of this invention. Even if the constituent 
elements of the character information storage terminal are 
implemented in software, the processing of this software 
is carried out according to the flowchart shown in this 
drawing. 

[0056] Television broadcast or other image 
information is input in frame image units in step 501, 
information concerning the input image is extracted in 
step 502, and the character information extracted in step 
502 is stored in step 503. 

[0057] In step 504, the character information stored in 
step 503 is provided to an image storage and playback 
terminal as necessary. A timing chart that describes step 
504 in detail is shown in the previously mentioned Figure 
5. 

[0058] As the method for extracting information 
concerning an image that is carried out in step 502, it may 
be done automatically using a predetermined method, it 
may be done by manual operation by the operator, or the 
results that have been obtained automatically may be 
revised and added to by manual operation by the operator. 

[0059] If the character extraction processing of step 
502 is to be carried out automatically, this can be done by, 
for example, combining and using the technology 
described in Kuwano et al., "Reduction of telop frame 
detection error using edge density" (Proceedings of the 
Society Conference of the Institute of Electronics, 
Information, and Communication Engineers (IEICE), D- 
12-12, September 1997), the technology described in 
Kuwano et al., "Extraction of low-resolution telop 
characters based on two-step line binarization" 
(Proceedings of the Society Conference of the Institute of 
Electronics, Information, and Communication Engineers 
(IEICE), D- 12-20, p. 207, September 2000), and the 
technology described in Mori et al., "Robust telop 
character recognition in video using background and 
foreground features and dynamic modified classifiers" 
(Transactions of the Institute of Electronics, Information, 
and Communication Engineers (IEICE), vol. J83-D-2, No. 
7, pp. 1658-1666, July 2000). 

[0060] Figure 7 is a flowchart for the image storage 
and playback showing the seventh embodiment of this 
invention. Even if the constituent elements of the image 
storage and playback terminal are implemented in 



software, the processing of this software is carried out 
according to the flowchart shown in this drawing. 

[0061] Image information input from a television 
broadcast in step 601 is stored in step 602. Storage of the 
image information that is input can be realized on hard 
disk, for example. 

[0062] In step 603, the user designates whether to play 
back stored image information. If it is to be played back, 
one goes to step 604; if nothing is to be played back, it 
comes to an end. 

[0063] In step 604, the user in playing back an image 
selects whether the character information concerning the 
image for playback shall be taken from the character 
information storage terminal. If character information is 
to be received, one goes to step 605; if index information 
is not to be received, one goes to step 609. 

[0064] In step 605, the image's character information 
is taken from the character information storage terminal 
and is read. Figure 5, which was referred to above, gives 
a timing chart that explains in detail the procedure until 
the receipt of character information is requested in step 
604 and the character information is read in step 605. 

[0065] In step 606, the user puts out a search request 
directed at the received character information, and in step 
607 a frame image in which the character corresponding 
to the search results are displayed is displayed as a 
representative image. 

[0066] The following two methods are possible for the 
user to make a search request in step 606 and for a 
representative image to be displayed in step 607. 

[0067] In the first method, the user designates the 
broadcast time of a television image in step 606, and in 
step 607 frame images of the scene in which characters 
present in the image during the broadcast time designated 
in step 606 from among the index information, are 
displayed in temporal sequence. 

[0068] When this is done, it is also possible to use 
attribute information of the characters to display the 
content of the image on the screen in a structured form. 
For example, by using the positional information of the 
characters and selecting headline characters of each news 
item in a news image, a representative image for each 
news item can be displayed, which enhances the 
overviewability of the images. 

[0069] Technology for selecting headline lettering for 
each news image among the above news images can be 
realized by using, for example, the technology described 
in patent application HI 1-253944 [1999]. 

[0070] This technology is explained with Figure 9. 
This technology offers a character string information 
extraction device and method that can inhibit the effects 
of contrast and noise in the background around character 
strings and accurately extract height and width 
information about character strings. 

[0071] On images stored in an image input and 
storage unit 1 1 that include lettering, 

character outline pixels whose brightness value 
changes suddenly are detected by a character outline 
detection unit 12, multiple pixels connected with locally 
uniform brightness are extracted as a character region by a 
character region extraction unit 13, and a rectangular 
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image that surrounds a character string is extracted by a 
character string rectangular extraction unit 14 using a 
character string rectangular extraction method, etc. For 
each line horizontally and vertically in this rectangular 
image, a character string information extraction unit 15 
determines the number of said character outline pixels 
detected and the number of extracted character regions, 
and if both have a high value, the line is taken as a 
character string line, and a rectangle surrounded by the 
uppermost and lowermost horizontal character string lines 
and the leftmost and rightmost vertical character string 
lines in said rectangular image is extracted as a new 
character string rectangle and is output to a result output 
unit 16. These control operations are carried out by a 
processing control unit 17. 

[0072] As the second search request method and 
representative image display method, in step 606 the user 
inputs by character codes a keyword concerning the 
desired image, and in step 607 character information that 
matches the keyword input in step 606 and the 
corresponding frame image are displayed from the 
character information. 

[0073] In step 608, the user selects the image he wants 
to play back from among the displayed representative 
images. This can be realized by, for example, 
manipulating a mouse on a computer screen. 

[0074] In step 609, the image is played back. As the 
playback method, if in step 604 the instruction is given 
not to use the character information of the image for 
playback, the image for playback is played back from the 
beginning, and if in step 604 the instruction is to use the 
index information, cuing playback is done beginning with 
the image selected in step 608. 

[0075] In step 610, the image played back in step 609 
is displayed on a computer screen, etc. 

[0076] Figure 8 shows, as the eighth embodiment of 
this invention, an example of character information stored 
by the character information storage terminals 2 and 3 in 
the first embodiment of this invention. This character 
information can be stored in, for example, a relational 
database. 

[0077] Figure 8(a) shows an example of character 
information, and Figure 8(b) is a frame image in the 
television image that corresponds to the character 
information in Figure 8(a). In Figure 8(a), 801 represents 
the date of broadcast of the television image that has been 
input, 802 represents the time of broadcast of the 
television image that has been input, 803 is channel 
information on the television image that has been input, 
and 804 is the number of the frame image in which the 
characters are displayed in the first set. 

[0078] Also, 805 is a numerical value showing the 
character display position of the upper-row character 
string among the characters in the frame image of Figure 
8(b). 810 is a numerical value showing the display 
position of the lower-row character string. 806 and 811 
are numerical values showing the tilt of the character 
string. 807 and 812 are numerical values showing the 
brightness information of the character string. 808 and 
813 are numerical values showing the length of the time 
slot in which the characters are displayed. 809 and 814 



show the result of converting the characters to character 
codes using character recognition technology. 

[0079] If character information is found among the 
character information stored in the character information 
storage terminals 2 and 3 that matches the time 
information and the input source identifier and other 
attribute information concerning the terminal that were 
transmitted from the image storage and playback terminals 
6 and 7, then the character information storage terminals 2 
and 3 transmit to the image storage and playback 
terminals 6 and 7 character index information as shown in 
Figure 7. In this way one can obtain on the side of the 
image storage and playback terminals 6 and 7 detailed 
information on the stored image information, and based 
on this, one can play back images from a preferred scene. 

[0080] Thus the character information provision 
system of the various embodiments of this invention is a 
system that consists of image storage and playback 
terminals that realize the function of efficient image 
playback by extracting character information from the 
image information distributed by broadcasting or electric 
distribution, storing the extracted character information, 
inputting as necessary the character information storage 
terminals [sic] and images to be provided, storing and 
playing back, and upon playback acquiring character 
information on the server. 

[0081] The character information provision system 
according to the above embodiments of this invention is 
not limited to the examples described in the above 
embodiments, and its parts other than its hardware parts 
such as input and output devices in the system, 
communication devices, computer CPU and memory, etc. 
can be built with software (programs) that realizes the 
same functions and can be stored on a disk device, etc., 
and as necessary it can be installed on a computer or 
downloaded to a computer over a network and can 
perform character information provision service. In 
addition, the programs that are built can be stored on a 
transportable recording medium such as floppy (registered 
trademark) disk (registered trademark), memory card, CD- 
ROM, etc. 

[0082] Above, representative embodiments of this 
invention have been described, but this invention is not 
limited to the above embodiments; within the scope of the 
patent claims, various modifications and applications can 
be made. 

[0083] 

[Effects of the invention] As explained above, the 
following effects are obtained with this invention. 

(1) In the character information storage terminal, 
image information including characters is input, character 
information concerning the image that has been input is 
acquired, the acquired character information is saved as 
an index for image viewing, and only the character 
information is provided to the image storage and playback 
terminal; thereby the capacity of the storage unit within 
the character information storage terminal and the burden 
on the device can be reduced, and a system can be 
realized just by using a low-bandwidth communication 
circuit for connection with the image storage and playback 
terminal. 



Unexamined patent 2002-199302 (P2002-199302A) 

(9) 



(2) In the image storage and playback terminal, just by 
storing images, when a stored image is to be viewed, by 
receiving the character information from the character 
information storage terminal via a communication circuit, 
efficient image viewing can be realized using the received 
character information, without doing processing for 
automatic acquisition of the character information by the 
image storage and playback device itself. 

[Brief Explanation of the Drawings] 

[Figure 1] This is a schematic block diagram of a 
character information provision system showing the first 
embodiment of this invention. 

[Figure 2] This is a timing chart of the time 
synchronization procedure showing the second 
embodiment of this invention. 

[Figure 3] This is a timing chart of the time 
synchronization procedure showing the third embodiment 
of this invention. 

[Figure 4] This is a timing chart of the time 
synchronization procedure showing the fourth 
embodiment of this invention. 

[Figure 5] This is a timing chart showing the fifth 
embodiment of this invention, from the character 
information storage terminal as far as the provision of 



character information to the image storage and playback 
terminal. 

[Figure 6] This is a processing flowchart in the 
character information storage terminal, showing the sixth 
embodiment of this invention. 

[Figure 7] This is a processing flowchart of the image 
storage and playback terminal, showing the seventh 
embodiment of this invention. 

[Figure 8] (a) and (b) are explanatory diagrams of the 
character information that is stored in the character 
information storage terminal, showing the eighth 
embodiment of this invention. 

[Figure 9] This is a block diagram for explaining the 
conventional technology for selecting headline lettering in 
a news image. 

[Explanation of the symbols] 

1,4, 5, 8... TV tuner 

2, 3 ... character information storage terminal 
6, 7 ... image storage and playback terminal 

9 ... communication circuit 

10 ... time information server terminal 
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1 TV tuner 

2 character information storage terminal 

3 character information storage terminal 

4 TV tuner 

5 TV tuner 

6 image storage and playback terminal 

7 image storage and playback terminal 

8 TV tuner 

10 time information server terminal 



Character information storage terminal, image storage 
and playback terminal 

Time information server terminal 
S101: time alignment request 
SI 02: communication connection 
time information request 
SI 03: time information transmission 
time information 
SI 04: communication cutoff 
SI 05: time information setting 
End End 
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[Figure 3] 
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[Figure 4] 
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Image storage and playback terminal 

Character information storage terminal 

S201 : time alignment request 

S202: communication connection 

time information request 

S203: time information transmission 

time information 

S204: communication cutoff 

S205: time information setting 

End 



End 
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S304 
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Character information storage terminal, image storage and playback terminal 
Character information storage terminal that serves as the basis for time 
information 

S301: time alignment request 

S302: communication connection 

time information request 

S303: time information transmission 

time information 

S304: transmission cutoff 

S305: time information setting 

End End 
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1^. 



Image storage and 
playback terminal 

Character information storage terminal 

S401 : character information request procedure 

S402: communication connection 

S403: time information communication procedure 

time information 

S404: character and communication procedure 
S405: character information communication 
procedure 

character information 
S406: communication cutoff 
End End 
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S502: character extraction 
S503: character extraction results 
storage 

S504: character provision 
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Start 
S601: image input 
S602: image storage 
S603: Playback? 
S604: Use character information? 
S605: character information read-in 
S606: character information search 
S607: representative image display 
S608: representative image designation 
S609: image playback 
S610: image display 
End 
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(a): Example of character index information 




(b): Frame image corresponding to the 
character index information of (a) 
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e»*wi^*j««*s«-r* Uf7y3 0 3) . 
[0044] mmmmnmrntteziisim^mMmmw 

to&<otttte&H*&ft<aj&mm*mmii. Rum 

7°3 0 4) . 

[0 0 4 5] ^Jtf#©S^£&£^*^©£^tif$8 

xmfe-rz (xf7^3 o 5) . 

[0 0 4 6] fcis, Vi"fn^©»*©l^*iJffif«iRJ(li* 

\z&i$i^<Dmmftm&mm?z>&oizisX*>gk\<\ 
[0047] muz^2titz*mw<Dmi<DMMj&mm 
(ox^mmmmyxT&iz&^x, jczmmwmm* 

2, 3^&***«B4S8*6, 7^aff[Hl^9^^L 

5 (ommmmmt Lxm 5 o^-r 3 h 

[0 0 4 8] efc##«pf£S$*6, 7ti5HTi?ffi 
?iJffl*(i*##HS^**6, 7m5a 

1) . i nmmmm9 $^utx 

^«$B#«5S*2, 3K8H£nS (Xf>^4 0 

2) . 

[0 0 4 9] fttt*J*35r$n*ttfcC, lfeflf!#aS± 

as* 6, 7T#astiT^?>s4M*©sft#ifS0ii^ 

-5 Ux-y7°4 0 3). 

[o o 5 o] 3ii wmznx^ 

6 jiff $ nfc M&Hf $1 t MiSf* <=> tifztt ft 

f5 (Xf77"4 0 4) . Z\<D£Z, Hyfc#«B±S8* 
6, 7 tX^ffif*#«**2. 3 <DWC, 

*)vmn<D&5tz\ism&¥ffev&%m\T*:tt j tomz 
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ftTV>fc*£ \Z t>«SS*&*£#5£0 *> CD K« t) jX 
[0 0 5 1 ] X^lf 2 , 311 ± 

7°4 0 5) d 

[0052] ftiitMiS^e, 7j5t^©x?««s 

*2, 3 &OID«>91ittllt^«WrSti« Uf7^4 0 io 
6) . 

[0053] vkmmmn±>$3i 6 . 7 a> s x^t**** 

£«*6, 7ftt«:^*WIWM|5t£2, 3fc*V»T»«l 

2IW*&l^*J«i:a[2li»7l^*J*i*-3#«tWr«X^** 
&Ut«*«S4SIII*6. 7K2lfrr*«k3K:*fiKbTt> 

[0 0 5 4] ZinC«tt3l*«#a[f|4«f*6, 

t»is#a^*2 > 3 cDm-p^<Dmn«iifRicR}irr« 

[0 0 5 5] 0 *5B9§©IB6©*W»18fl£J:5 
©SiJ5!cS*SV7 h^iTT^ILitil^Cfc, Z.CD 

77 hfi7©jl!iiraai:^Uft7n-f t-M:ft 
oTfrfcns. 

[0 0 5 6 ] xUe*2!fc£©&*W«a**xyy5 0 

2TAjjVimizm?z>mmtf$:¥tL~cmtii2ft, 
sarrs. 

[0 0 5 7 ] 7f-yy5 0 4 TUT^y 7°5 0 3 T#« 

- MiittMcD^5T^LfcfcCDT*So 
[0 0 5 8] ^ Xxy7°5 0 2Tfffcn*WHRCH 

* CD V - i Timft IZ «t o TfTtinT fe i£%£ T A ^ 

hcDtrz* 

[0 0 5 9] 7f>y^5 0 2©X?JflfflfflaSiKjWK so 
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t-yXrAVU-'fXf^^. D-12-12, 19 

ilff^lflS /7fAVii-'fIf-f^ D-12- 
20, p. 207, 2000^9 CfltSntlS 

Iff, RtfjRflj rws • £^0»tt#«i»«*fciEl*»j|JI 

^laSCMr, vol. J83-D2, No. 7, pp. 1 
6 5 8 - 1 6 6 6. 2 0 0 0^7^) 

[0060] H7ii ^m<Dm7 cDmmmmm^^r 

I(D7 7h7i7©Il(t Billf Lt7P-ft- 

[0 0 6 1 ] rl/t'ltl^e.7f77'6 0 1 IrioTA 
^J^nfclftiiSitt, ^fy7 , 6 0 2l:^Tf»Sft 

£ & £* T*§I"r 5 £ £ #T £ 5 „ 
[0 0 6 2] Xry7'6 0 3iI^Ut 1 *iJffl#H#«$ 

«^lXf7 7'6 0 4'\, S^L/i^«^ti^7f So 
[0 0 6 3] Xf7^6 0 4TI1, ?iJffl#«Sft«SrS* 

tflXS&l/^'&teT.T^.y 7°6 0 9^-5. 

[0064] x^-7 7°6o 5 m 

7°6 0 4T£^tt^cD§tfI!5D£g*U Xf-^6 0 
5T-X?1**^^*iAt;ST«#l«*PlfflfClK0J^S^ 
■f 5 > iff- f - h fi Bif i£ <D 0 5 L CD T $> 3 . 

[0 0 6 5] 7>fy?6 0 6tli SttlXo ffi 
KttUfM##lftfSE#StiJU Xr-^6 0 7 t'*3Vi 

[0 0 6 6] Xf 7^6 0 6 \z£>tfZ>mm%CD& 

[0 0 6 7 ] -Oglt Xt--7 7°6 0 6 IZ&^T, mm 

%\3.TUtLVkm<Dtkm.mm$:mi£v, xf7 7°6 0 7 c 

^TH-O^Xlt «©+^6»^^-y7° 6 0 6 Tfgjg 

£ n^^n#pfli*icD*#+ n#ftt4i?^g ns 

- > CD 7 U~ A Htt * Bf W)l C S^-T -5 fc CD T $> 2> . 
[0 0 6 8 ] ^C73^, S:^CDStt1tffi$flJfflLTft#l*I 
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[0 0 6 9 ] ±)£c7)-n.-X^ + <Z)#-i-X^CD 

^wb©x^saiR-r*ftWfi, mm^-i i - 
2 5 3 9 4 4^j i:n«snTi^ftK?wn;i 
tsist^-So 

[0 0 7 0 ] H9t«fcD. 

[0071] H#A^7fBtt^i i izimzntz, x^* 
^tsmmzML, ^mmu^^i 2\zx^onmmm 

mtvxtamL. j&nt&mm&i 4iz^vic^mm 
ic^wm mmmffl 1 5 n» ^o^H#»(fT*¥* 

T#¥# IbJ CO* t> ± (75 5 -f > £ « fe T<D y 

x^jsk yizm^ti^mm^mrzux^mmmt vx 
fflajL, isma^i 6A.m^n 0 ^necofwwi® 

[0 0 7 2] -t3B©*lSS**jS, fttfft&H*©a 
«^j*tl/Tfi, 7fy^6 0 6l:*ViT, ?iJffl#tt/5? 

f'y^6 0 7Tit 3C^»m<D*fr<E>Xy-y76 0 6 T 

[0 0 7 3] Xf7^6 0 8T(J. WB^te^aftfc 

«3 > fcf a. - 9 iBS±C0 v £ X €T -5 o 

[0 0 7 4] 7f-7 7 - 6 0 9Tlt WiSStStlS. 
Bt*StlTIJXf7^6 0 4 7?S±*f*©lfttt(OX 

5teSW>SB£U Xf7^6 0 4T-T>T ; ^Xtf#$« 
ffl-f-3£Jt^L£«£te, Xy-y-fQ 0 8T8#jan£: 

[0 0 7 5] Xf7^6 1 OtH Xry-fS 0 9TB 

£ a ntz vkm £ a > t° a - * mm ±& if c is^t 5 . 

[0 0 7 6 ] *5SWO*l Wg|Jfi»IBWf;45^5X^tff 
#T#3, 

[0 0 7 7] 08 (a) tt^«*<D— WSitfeOt 
&D. 08 (b) (208 (a) 0X?||i:jl*)St5f 
l/t*^# + (D7l/-AH«T$.S„ 08 (a) T801 so 
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ttA^anfc^lxk'^#C0ftjiB2r^L> 8 0 2 (iAX7 
an/t5 l Hflft#©JS[2IRp*JSSL'. 8 0 3 (JAA^n 
tzy-V\d^(D^^y^.)Vmnxh'0, 8 0 4ti-#» 
rtTX^gjj* a ftT lr> 4 7 V- AHflt0#*t-C& -5 o 

[0 0 7 8] 8 0 5 H0 8 (b) ©XU-AHHgi 

*COX* CD 3 *. iacOSi^JcOX^S^iM^^-rifcfl 

t^s. 8 1 o HTaoi^jioasffiis^tfttT 

»S. 8 0 6, 8 1 1 ttX^©«***f»MT* 

8 0 7, 8 1 2tt£^<Z)IJIi*S©|t*£*-$-JJc£[ 
T&£„ 8 0 8, 8 1 3 ttX^i^atlTl^RfEW 
<Dfta£^-fSfcffiT&3o 8 0 9, 8 1 4 teX^SrX^ 

[0079] x^«$g#a«*2, 3T#fflatL^s:* 
<Dmm^Aijm<Dmwfr®<D*simzm-rzm)tmm&~ 

2, 3H0 7(^^TJ;5^^-r>7 :r i7Xtf$SS^## 

s*6, 7#jTtt, ^aanfc^tffficopM^isss: 
[0080] z.<D£?iz*%w(Dm*(Dmffimmm<Dj: 

a#u sat. s£u mkTzmizy— 

[0 0 8 1 ] ±iB©*»M©**»IBWfc«t4X^«« 

JfftvXxA«> ±Eo*il^lg«aiTiawatife«s|{rK 

TAMX, n>tfa-^fflCPU J ^^ ; £'J^i:£DA-H 
LT£i*, <&mzfo\ST=L>}d3.-?lZl >Z b-Jl 

-So ae.c iisstifcT'n^^AS^D^ tf- 

iffi*) x^X^7 (ae^ao « ^tUA-K, CD-R 
[0 0 8 2] £*±, *58MOft*»34^J6}g»«f!|SttM 

Tab -5. 

[0 0 8 3 ] 
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